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User  Accuracy  Requirements 
For  Voice  Recognition 


I.  INTRODUCTION 

Within  the  last  ten  years  voice  recognition  technology 
has  grown  from  a  laboratory  research  endeavor  to  a  useful 
and  economic,  computer  human  interface  tool.  The  aguipment 
available  today  is  relatively  inexpensive,  compact,  and 
accurate  as  evidenced  by  numerous  applications,  both 
industrial  and  military,  which  successfully  use  this  input 
methodology.  Improvements  in  the  technology  are  still  being 
made.  Three  major  areas  for  continued  research  in  voice 
input  to  computers  are  as  follows. 

1.  User  independence 

2.  Continuous  speech 

3.  Vocabulary  capability  of  10,003  to  12,000  words 

The  literature  indicates  that  the  solution  or  ax  least 
incremental  breakthroughs  to  these  three  research  areas  are 
just  a  few  years  away.  Hopefully,  the  goal  of  manufacturers 
and  researchers  is  to  develop  and  produce  systems  with  all 
or  some  of  the  properties  listed  above  which  have  the  best 
accuracy  rate  possible.  A  question  arises  with  this  goal  in- 
mind.  At  what  accuracy  rate  should  the  system  be  made 
available  to  users?  If  an  accuracy  rate  of  95*  is 
acceptable  why  not  use  those  systems  while  new  and  better 
algorithms  are  being  developed?  Conversly,  if  95*  accuracy 
rate  is  not  acceptable  and  will  give  voice  recognition  a  bad 
reputation  among  management  personnel  and  users  it  should  be 
held  back  until  an  acceptable  rate  can  be  achieved.  A 
measure  of  where  this  accuracy  threshold  is  could  be  of 
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great  use  to  researchers  and  system  manufacturers. 
Therefore,  the  question  investigated  was  how  poor  must  a 
voice  recognition  system  be  before  the  user  becomes 
frustrated  with  the  error  rate  and  will  choose  not  to  use 
the  system. 

A  required  accuracy  rate  will  depend  on  the  task  for 
which  the  system  is  intended  to  be  used.  For  example,  if 
voice  recognition  is  to  be  used  for  the  input  of  guidance 
parameters  and  launch  sequence  commands  in  a  computerized 
missile  delivery  system,  anything  less  than  99 +%  accuracy 
would  be  unacceptable  even  under  the  high  stress  situations 
most  likely  existing  during  the  input  process,  on  the  other 
hand,  there  are  numerous  tasks  which  can  be  labeled  as  non- 
critical  where  voice  recognition  errors  can  be  tolerated  and 
in  fact  now  occur  frequently  using  more  conventional  input 
methods  such  as  a  keyboard.  A  typical  task  in  this  category 
would  be  information  retrieval  such  as  a  stock  broker 
obtaining  stock  information  for  a  client,  or  an  airline 
reservationist  retrieving  flight  information  for  a  customer. 
Neither  of  these  tasks  are  particularly  critical  in  nature. 
If  an  occasional  error  is  made  during  the  input  process  it 
is  easily  corrected  and  the  process  repeated  without  any 
damage  to  the  system  or  loss  in  revenue,  profit  or  system 
efficiency. 

An  experiment  was  developed  to  study  the  user 
acceptabiltiy  question  with  the  objective  of  determining  a 
user  required  accuracy  rate.  A  VET/2  voice  recognition  unit 
manufactured  by  Scott  Instrument  Inc.  of  Denton,  Texas  was 
used  in  conjunction  with  a  Basic  software  program  created  on 
an  Apple  II  Plus  microcomputer.  The  experiment's  subjects 
were  asked  to  read  an  ordered  list  of  words  into  the  voice 
recognition  unit  and  observe  whether  the  word  was  recognized 
properly.  This  same  ordered  list  of  words  was  stored  in 
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computer  memory.  Each  subject  was  led  to  believe  the  system 
was  in  a  recognition  mode,  but  in  fact,  the  software  program 
waited  for  a  verbal  input  and  then  drew  a  random  number  to 
determine  if  the  spoken  input  should  be  displayed  as  correct 
recognition,  misrecognition  or  nonrecognition.  If  the 
random  number  determined  it  was  to  be  a  correct  recognition 
the  next  word  or.  the  ordered  list  was  displayed.  if  a 
misrecognition  was  to  occur  an  alternate  word  was  displayed, 
and  a  question  mark  was  displayed  if  the  random  number 
determined  a  nonrecognition  was  to  appear.  Ir.  other  words, 
the  program  can  be  viewed  as  a  voice  actuated  system  where 
any  verbal  response  would  trigger  the  recognizer,  but  a 
random  number  would  determine  the  output,  not  the 
recognition  algorithms.  ks  long  as  the  subjects  continued 
to  read  the  ordered  list  of  words,  it  appeared  as  if  word 
recognition  was  being  accomplished.  By  varying  the  random 
number  test  the  subject’s  perceived  accuracy  rate  could  be 
controlled.  The  errors  were  recorded,  and  each  subject  was 
asked  to  complete  a  questionnaire.  The  questions  were 
designed  to  indicate  when  a  frustration  level  was  reached 
due  to  recognition  inaccuracy. 

The  experiment  did  not  lead  to  the  desired  goals  or 
results.  This  paper  will  cover  the  experiment  and  why  the 
actual  results  were  different  than  predicted.  First,  the 
report  will  describe  the  computer  software  program  which  was 
created  to  vary  the  perceived  recognition  rate  during  the 
experiment.  Next,  the  experimental  design  will  be 
discussed.  The  entire  design  was  not  implemented  because 
preliminary  data  analysis  indicated  the  desired  results  were 
not  being  obtained.  The  method  used  to  implement  the 
experimental  design  will  also  be  discussed.  The  third 
section  of  the  report  will  cover  the  preliminary  data 
analysis  and  summary  of  user  responses  to  the  experiment. 
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Finally,  conclusions  will  be  drawn  as  to  why  +  he  experiment 
did  not  resolve  the  question  at  hand,  and  recommendations 
will  be  presented  for  future  research  concerning  the 
question  of  "How  good  should  voice  recognition  equipmant 
be?". 


II.  PROGRAM  SOFTWARE  DESCRIPTION 


Two  computer  software  programs  were  written  for  use  in 
this  experiment.  The  first  program  was  used  to  create  three 
databases  consisting  of  words  to  be  used  for  the  planned 
experimental  design.  Since  this  program  was  used  only  as  an 
aid  for  the  database  preparation,  it  will  not  be  discussed. 
A  copy  of  the  program  and  the  databases  of  words  are 
attached  in  Appendix  A  and  Appendix  B  respectively.  The 
second  program  created  was  used  to  alter  the  perceived 
recognition  accuracy  of  the  Scott  Instrument  VET/2 
recognizer.  It  was  written  in  Applesoft  Basic  and  is 
included  as  Appendix  C.  The  following  description  explains 
the  program  logic  using  the  program  line  numbers  for 
reference. 

Line  10  dimensions  a  character  array  called  w$  (200,2). 
During  the  experiment,  this  array  held  the  200  vocabulary 
words  used  for  the  experiment.  W$(I,1)  held  the  word  which 
was  to  be  spoken,  and  W$(I,2)  held  a  sound  alike  word.  The 
use  of  this  sound  alike  word  will  be  explained  later.  The 
array  F$(3)  held  the  name  of  the  three  word  databases  which 
were  available  for  use.  These  databases  were  named: 

1.  COMPUTER,  designed  to  be  used  by  users  of  a 
text  editor  on  the  IBM  3033  computer  at  the  Naval 
Postgraduate  School, 

2.  STOCK,  designed  to  be  used  by  stockbrokers,  and 

3.  AIRLINE,  designed  to  b?  used  by  airline 
reservationists  or  travel  agents. 

The  array  P(4)  held  the  four  different  probabilities 
associated  with  the  planned  recognition  rates  of  99%,  95%, 

90% ,  and  85%. 
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line  20  sets  the  variable  D$  to  a  control  D  which  is  used 
for  file  manipulation  on  the  Up  pie  computer. 

lines  30  and  «0  are  data  assignment  statements  for  the  F$ 
and  P  arrays. 

line  45  assigns  to  the  character  variable  M$  a  series  of 
five  blank  spaces.  MS  is  used  for  print  control  or  print 
spacing . 

line  50  is  an  output  statement  asking  the  user  what 
database  is  to  be  used. 

line  60  accepts  as  an  input  the  number  of  the  desired 
database.  This  number  is  placed  in  the  variable  A. 

line  70  opens  the  correct  database  file. 

line  80  sets  the  read  device  to  the  appropriate  database 

file. 

line  90  reads  in  the  200  word  vocabulary  and  their  sound 
alikes  into  the  v$  array. 

line  100  closes  the  input  database  file. 

line  105  asks  for  a  random  number  generator  seed  and 
places  the  integer  response  into  the  variable  called  is. 

line  110  asks  for  an  algorithm  number,  which  in  effect  is 
the  array  index  of  the  desired  accuracy  rate.  The  question 
in  line  110  was  stated  in  such  a  manner  so  that  experiment 
participants  would  assume  different  word  recognition 
algorithms  were  being  tested. 

line  120  places  the  algorithm  number  in  the  variable  B, 
and  if  B  is  less  than  zero,  the  program  is  stopped. 

lines  130  through  160  call  the  Scott  instrument  voice 
recognition  subroutines  used  to  load  the  voice  patterns  into 
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■emery,  and  initialize  the  recognition  unit.  The  voice 
patterns,  although  not  used  for  recognition  purposes,  were 
neccessary  for  the  proper  operation  of  the  recognition  unit. 

Line  170  prints  a  header  announcing  the  practice  session 
of  10  words.  During  this  practice  session  the  use  of  voice 
recognition  eguipmer.t  was  explained  to  the  experimen* 
participant.  The  explanation  given  to  each  participant  will 
be  described  in  detail  later. 

Lines  180  through  200  create  a  program  loop.  The 
variable  I  is  used  as  an  indexing  variable.  This  indexing 
variable  is  first  set  to  1,  and  the  first  word  of  the 
vocabulary  database  is  printed  on  the  Apple  computer- 

display.  This  display  is  used  .as  a  prompt  to  the 

experiment's  participants  for  the  word  they  are  to  speak. 
After  the  word  is  displayed  the  program  is  transferred  to 
the  recognizer's  subroutine  which  will  accept  a  verbal 
input.  After  the  recognizer  accepts  the  voice  input  the 
subroutine  returns  control  to  this  main  program.  After  the 
acceptance  of  the  voice  response  an  artificial  delay  is 
created  by  the  "FOR  Z  ..."  statement.  This  delay  was 

neccessary  to  provide  a  capability  of  stopping  program 
execution  if  the  participant  made  an  error  which  could  lead 
them  to  believe  the  program  was  not  actually  recognizing 
their  voice.  The  delay  provided  the  time  for  +he 

experimenter  to  stop  the  program  before  the  "recognized" 
word  was  displayed.  After  the  delay  a  subroutine,  which 
will  be  described  later,  is  called  to  determine  whether  a 
correct  response,  a  sound  alike  mistake,  a  random  mistake, 
or  a  nonrecognition  response  should  be  displayed  to  the 
participant.  After  the  response  is  displayed  the  index 
variable,  I,  is  checked  to  determine  if  the  test  practice 
session  is  over. 
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line  210  sets  a  series  of  counters  to  zero.  These 
counters  keep  track  of  the  number  of  non  recognitions 
presented  to  the  participant  (Q ) ,  the  number  of  sound  alike 
misrecognitions  presented  (S> ,  and  the  number  of  nonsense 
Slisrecognition  responses  (N)  .  The  variable  T  is  calculated 
at  the  end  of  each  particpant's  pass  through  the  200  word 
vocabulary  list  and  holds  the  accuracy  rate  actually 
presented  to  the  subject. 

Lines  220  through  250  comprise  another  program  loop.  The 
logic  is  similar  to  lines  180  through  200  except  the  entire 
200  word  vocabulary  is  sequentially  displayed. 

Lines  260,  270  and  280  respectively  display  statements 
thanking  the  participants,  calculating  the  actual  accuracy 
rate  presented,  and  displaying  the  accuracy  rate  and  all 
counters  in  a  coded  form. 

Line  290  sends  control  of  the  program  back  to  the 
question  asking  which  algorithm  should  be  used  (Line  110), 
and  the  program  is  ready  for  the  next  participant. 

Line  300  is  the  first  line  of  the  subroutine  which  will 
calculate  whether  the  response  which  is  to  be  shown  to  the 
participant  is  a  correct  recognition  response  or  one  of  the 
three  possible  error  responses.  A  random  number  is  drawn. 
If  the  random  number  is  greater  than  the  accuracy  rate  which 
is  presently  being  simulated  the  program  will  branch  to  the 
statements  neccessary  to  calculate  the  type  of  error  which 
should  be  presented. 

Line  310  is  executed  if  a  correctly  recognized  response 
is  to  be  displayed.  The  print  statement  will  first  prin*- 
the  variable  containing  the  blank  spaces,  and  then  the 
correctly  recogniz®d  word.  This  was  done  so  the  recognized 
word  was  displayed  further  to  the  right  on  the  Apple  screen 


8 


than  the  word  which  was  output  as  a  prompt.  Transfer  ther. 
is  passed  to  statement  400. 

Lines  320  trhough  340  are  used  to  determine  the  type  of 
error  that  should  be  displayed  given  that  the  present  word 
is  to  be  perceived  as  an  error  ,'j  the  participant.  A  random 
number  is  drawn,  and  if  it  is  less  than  .33  the  error  is 
considered  a  sound  alike  error  and  control  is  passed  to  line 
390.  If  the  random  number,  R,  is  between  .33  and  .56,  it  is 
considered  a  random  or  nonsense  error  and  control  is  passed 
to  line  360.  Finally,  if  the  random  number  is  greater  than 
.66,  it  is  considered  a  nonrecognition.  Therefore,  the 
three  types  of  errors  are  equally  likely.  In  previous  voice 
recognition  studies,  Poock (  1980) ,  Poock(1981),  and 
Jay  (1981),  error  rates  of  about  1.8%  have  been  consistently 
experienced  with  a  Threshold  Technology  Inc. ,  Delran  New 
Jersy,  model  600  voice  recognizer.  Ir  these  studies  non¬ 
recognitions  consisted  of  31%  to  35%  of  the  total  recorded 
errors.  There  were  no  statistics  available  on  the 
percentage  of  misrecogn itions  which  could  be  considered 
sound  alikes  or  non-int uitive  confusing  misrecognitions. 
For  this  reason  it  was  assumed  that  misrecognitions  should 
be  egually  divided  between  the  sound  alike  possibilities  and 
the  random  error  possibilities.  Therefore,  all  three  error 
types  were  programmed  to  occur  with  equally  likely 
probabilities. 

Line  350  is  executed  if  a  non  recognition  is  +o  occur. 
It  again  prints  the  variable,  f!$,  which  contains  blanks  and 
then  a  question  mark,  ?,  representing  the  Scott  Instrument 
convention  for  a  non-recognition.  The  counter  for  non¬ 
recognitions  is  increased  by  one  and  the  program  is  then 
transferred  to  line  400. 

Line  360  through  380  determine  a  random  word  response  for 
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the  random  misrecogr.ition  case.  A  random  number  is  drawn 
and  converted  into  a  random  integer  between  1  and  200.  Next 
it  is  checked  to  ascertain  that  the  random  integer  generated 
is  not  the  word  which  is  to  be  misrecogni zed.  If  this  check 
had  not  been  made,  a  misr ecogni tion  could  have  been  recorded 
but  the  participant  would  have  in  fact  seen  the  correct 
response.  The  randomly  selected  word  is  printed  on  the 
display  in  the  same  manner  as  a  properly  recognized  word, 
and  the  nonsense  word  counter  is  incremented.  Again  control 
is  then  passed  to  statement  400. 

line  390  is  executed  if  the  random  number,  R,  indicates 
that  this  incorrect  recognition  should  be  a  sound  alike  and 
the  second  word  in  the  W$  array  is  printed  out.  These  words 
have  been  selected  in  such  away  that  an  average  user  would 
conclude  that  it  was  an  understandable  or  likely  recognition 
error.  The  sound  alike  counter  is  incremented. 

line  400  is  the  last  line  of  the  subroutine.  Again  a  loop 
is  added  to  produce  a  delay.  The  index  used  in  the  program 
Ioof  consisting  of  statements  220  through  260  is 
incremented.  A  blank  line  is  displayed  for  readablity  and 
the  subroutine  returns  to  the  main  program  to  print  the  next 
word  in  the  200  word  vocabulary  list. 

The  next  section  will  describe  how  this  proqram  was  used, 
and  the  reaction  of  the  experiment’s  participants  as  to  the 
believability  that  a  voice  recognizer  was  being  tested. 
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III.  EXPERIMENTAL  DESIGN 

It  has  already  been  mentioned  that  ■t-hree  databases  w«re 
formed  for  the  research  experiment.  The  plan  was  to  rur. 
three  groups  of  100  people  each  through  one  of  the 
databases.  In  other  words,  100  stockbrokers  or  investment 
counselors  would  say  the  200  words  associated  with  their 
profession,  and  then  rate  the  acceptability  of  the 
equipment.  Likewise,  100  airline  reservationists  or  travel 
agents  were  to  use  the  airline  reser vationist  word  list,  and 
100  students  would  use  the  200  words  associated  with  the  IBM 
3033  text  editor  program. 

A  word  list  of  200  words  was  used  for  two  reasons.  First 
it  was  decided  that  going  through  a  list  of  100  words  went 
too  fast,  and  the  subjects  would  not  get  a  good  feeling  for 
the  accuracy  rate.  Using  300  words  was  definitely  out  of 
the  guestion  because  of  the  time  involved  in  conducting  the 
experiment  for  the  number  of  planned  subjects,  and  because 
of  a  possible  boredom  factor  which  could  complicate  the 
subjects  perception  of  the  system.  The  median  seemed  like  a 
reasonable  choice.  The  vocabulary  size  of  200  words  also 
satisfied  a  second  criteria.  That  was  the  desire  to  get 
user  frustration  information  at  a  more  accurate  level  than 
every  1  percentage  point.  At  least  the  200  words  would  give 
ratings  at  every  one  half  percentage  point. 

During  the  experiment  it  became  apparent  that  a 
frustration  level  was  not  being  achieved  or  at  least 
measured.  Therefore,  after  the  first  fourth  of  the 


experiment  some 

preliminary  data 

analysis 

was  done,  and 

the 

results 

showed 

that 

little  was 

being 

learned  about 

user 

accuracy  needs. 

It 

was  decided 

to  stop 

the  experiment 

and 

report 

on  what 

had 

beer,  done 

to  date. 

A  total  of 

78 
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subjects  participated  in  the  test.  The  intent  of  this 
section  of  the  paper  is  to  explain  how  the  subjects  were 
introduced  to  the  experiment  and  how  they  reacted  to  the 
recognition  system.  This  section  will  also  discuss  the 
design  of  the  questionnaire. 

The  subjects  were  students,  staff  and  faculty  members  at 
The  Naval  Postgraduate  School  in  Monterey,  California.  They 
were  all  volunteers  and  between  the  ages  of  25  and  49.  The 
entire  explanation  and  experiment  took  between  10  and  15 
minutes  per  subject. 

When  a  subject  arrived  at  the  experiment  sits,  it  was 
explained  that  some  new  user  independent  voice  recognition 
algorithms  were  being  tested.  The  Scott  Instrument  voice 
recognition  device  was  covered  to  preclude  the  subjects  from 
getting  the  wrong  impression  of  its  capability.  The 
algorithm  number  was  entered  into  the  system  in  front  of  the 
subject  as  the  idea  of  different  algorithms  was  being 
explained . 

Some  of  the  subjects  had  used  voice  recognition  eguipmen* 
on  previous  voice  experiments.  These  students  needed  little 
practice,  but  still  went  through  the  ten  practice  words. 
The  practice  words  were  used  as  the  teaching  device  for 
those  subjects  who  had  not  used  the  equipment  before.  The 
need  to  speak  a  phrase  as  a  continuous  flow  of  speech  was 
explained,  as  was  the  explanation  of  the  meaning  of  a 
question  mark  (?)  when  it  appeared  on  the  display.  If  the 
subject  showed  an  interest  in  the  machine's  capability 
during  the  the  practice  session,  their  questions  were 
postponed  until  after  they  had  answered  ^he  questionnaire. 

The  subjects  were  asked  to  ignore  all  the  errors  (if  any) 
which  occured  during  their  practice  session.  It  was 
explained  that  some  artificial  intelligence  algorithms  were 
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being  employed  and  the  system  was  select inq  characteristics 
of  their  voice  for  use  in  the  Bain  portion  of  the 
experiaent. 

As  the  subjects  were  going  through  the  200  word  list,  the 
experiaent  was  stopped  each  tine  an  error  occured.  The 
error  was  pointed  out  to  the  subject  and  a  short  explanation 
was  given  as  to  what  that  error  would  do  if  the  true  tex+ 
editor  was  employed.  The  idea  for  stopping  +he  program  was 
two  fold.  First  it  was  noticed  during  preliminary  program 
testing  that  some  people  started  to  read  the  words  on  the 
display  and  weren't  watching  the  recognized  word  displayed. 
In  ether  words,  the  experiment  bored  +hem  and  they  weren't 
always  aware  of  the  errors.  At  first  this  had  been  solved 
by  placing  the  recognized  word  directly  under  the  prompt 
word.  Unfortunately  this  solution  caused  another  problem. 
The  subjects  who  participated  in  the  preliminary  testing 
started  to  say  the  recognized  word  instead  of  the  following 
prompt  word  which  had  a  devastating  effect  on  the  user's 
confidence  that  the  system  was  recognizing  their  voice. 
Therefore,  the  recognized  word  was  placed  to  the  right  of 
the  prompt  word,  and  each  error  was  pointed  out  tc  the 
subject.  The  second  reason  for  stopping  the  proqram  was  to 
delay  the  subject  in  completing  the  experiment.  It  was 
hoped  that  the  idea  of  an  error  slowing  them  down  would 
transfer  to  their  perception  of  how  the  system  would  work  in 
a  real  environment.  The  plan  was  to  slow  them  down  thus 
creating  a  frustration  level  which  was  to  be  measured. 

There  were  numerous  times  even  with  the  precautions  taken 
where  the  subject  read  the  wrong  word,  or  started  to  make  a 
comment  without  the  microphone  being  turned  off.  This  led 
to  a  recognition  which  in  the  majority  of  cases  was  correct, 
when  the  spoken  utterance  was  obviously  incorrect.  The 
experimenter's  finger  was  always  kept  lightly  on  the  Apple 
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computer's  space  bar.  By  depressing  the  bar  the  microphone 
and  voice  recognition  system  were  deactivated.  The  majority 
of  the  time,  the  experimenter  depressed  the  space  bar  soon 
enough  to  avoid  a  correct  recognition  of  an  incorrect  voice 
input.  This  was  the  major  reason  for  the  delay  loop 
explained  earlier  in  the  program  software  description.  If 
the  space  bar  was  not  depressed  soon  enough,  and  the  correct 
response  appeared  on  the  display,  the  subject  was  told  that 
the  software  program  had  been  developed  to  do  its  own  data 
collection  automatically.  Furthermore,  it  was  explained 
that  the  experiment  was  interested  ir.  only  recognizer  errors 
not  the  human  errors  which  will  always  occur  with  a  voice 
system.  Therefore,  it  was  explained,  that  pressing  the 
space  bar  was  an  automatic  override,  and  no  matter  what  was 
said  the  automatic  data  collection  routine  would  count  it  as 
a  correct  recognition.  This  explanation  seemed  to  satisfy 
everyone,  who  encountered  the  situation. 

After  the  subject  had  completed  the  200  words,  he  or  she 
was  asked  to  fill  out  a  two  page  questionnaire.  This 
questionnaire  is  attached  as  Appendix  D.  Questions  3,4  and 
5  created  the  data  which  was  of  most  importance  to  the 
experiment  and  was  the  measurement  of  user  acceptability. 
The  sets  of  response  alternatives,  for  these  three 
questions,  were  taken  from  an  Army  Research  Institute 
publication  on  questionnaire  construction  (19761.  The 
responses  have  been  tested  and  shown  to  have  mean  scaling 
factors  at  least  one  standard  deviation  away  from  each  other 
while  maintaining  the  parallel  wording.  Question  9, 
concerning  the  part  of  the  country  the  subject  grew  up  in, 
had  nothing  to  do  with  the  experiment  that  was  being 
conducted,  but  was  included  to  make  the  experiment  about 
testing  user  independent  algorithms  more  believable  to  the 
participants.  The  remaining  guestions  are  self  explanatory. 
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and  will  be  reviewed  in  detail  when  discussing  the  analysis 
of  the  data  collected. 

After  the  questionnaire  was  completed  the  subject  was 
free  to  ask  questions  about  the  system.  They  were  led  to 
believe  that  all  algorithms  had  been  created  at  the  School, 
and  that  the  technology  was  not  commercially  ready  because 
of  the  extremely  large  amount  of  core  needed  to  run  the 
system.  This  was  done  to  insure  that  the  believability  in 
the  system  would  not  decrease  as  students  talked  about  the 
experiment.  On  the  other  hand,  it  was  not  the  intent  of  the 
experiment  to  lead  the  subjects  to  believe  that  voice 
recognition  capabilities  were  beyond  the  present  state  of 
the  art. 

The  experiment  was  a  total  success  as  far  as  the 
believability  of  the  system  was  concerned.  There  »°re  a 
couple  of  instances  when  the  random  number  generator 
cooperated  fully.  For  example,  a  subject  asked  a  question 
which  triggered  a  response  and  the  random  number  generator 
created  a  non recog nit ion .  In  another  case  a  word  was 
misrecogn ized  in  the  test  sequence  and  the  same  word  was 
misrecognized  during  the  experiment,  both  times  the 
mi srecognition  was  the  sound  alike  word.  There  were 
subjects  who  tried  to  analyze  the  system  and  hypothesized 
why  the  system  did  not  recognize  them  correctly.  One 
subject  was  convinced  that  any  word  with  an  "S"  sound  would 
not  be  recognized  properly  because  he  tended  to  slur  his  "SM 
sound.  There  were  only  two  subjects  out  of  the  78  tested 
who  mentioned  the  fact  that  they  doubted  the  system  was 
recognizing  their  voice.  This  fact  was  noted  on  their 
questionnaire  after  they  left  the  laboratory  area. 
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IV.  DATA  ANALYSIS 


Appendix  E  and  F  present  the  data  collected.  Appendix  E 
contains  the  raw  data  collected,  while  Appendix  F  has  the 
data  in  the  ranked  form.  All  of  the  data  analysis  used 
nonparametric  statistics  methodologies  based  on  ranks. 

The  first  column  of  data  in  Appendix  E  contains  the  total 
number  of  errors  the  user  observed.  This  number  is  the  sum 
of  nonrecognitions  and  both  types  of  misrecognitions.  The 
second  column  contains  the  total  number  of  misrecognitions 
which  is  the  sum  of  the  sound  alike  errors  and  nonsense 
errors  which  were  observed  by  the  subject.  Columns  3,4,  and 
5  are  the  individual  error  totals  for  nonrecognitions,  sound 
alike  misrecognnitions,  and  random  misrecognitions, 
respectively.  Column  6  is  the  age  of  the  subject,  and 
column  7  is  the  average  number  of  hours  the  subject  spends 
using  a  computer  terminal  each  day.  The  average  number  of 
hours  spent  at  a  terminal  were  considered  important  for  the 
stock  brokers  and  airline  reservationists,  but  had  little 
meaning  for  the  students  who  became  the  only  participants  in 
the  experiment.  Therefore,  this  data  will  no*  be  used  in 
the  analysis,  but  is  presented  for  completeness. 

The  next  two  columns,  column  8  and  9,  are  the  answers  to 
the  questions  on  whether  the  subject  had  ever  seen  or  used 
voice  recognition  equipment.  These  answers  are  coded  with  a 
1  representing  an  answer  of  '•yes",  and  a  2  representing  a 
"no"  response.  Columns  10,  11,  and  12  are  the  responses  to 
the  questions  dealing  with  user  acceptability.  These  are 
also  coded,  from  a  1  meaning  a  poor  acceptability  response 
to  5  for  a  high  acceptability  response.  Column  1 3  is  the 
sum  of  the  responses  to  the  three  questions.  Since  *he 
questions  each  had  parallel  wording  the  sum  of  the  answers 
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for  the  individual  questions  was  used  foe  the  data  analysis. 
This  method  gave  a  more  accurate  numeration  of  the 
acceptability  level  for  each  subject. 

Column  14  and  15  are  the  data  collected  on  the  subject's 
perception  of  his  or  her  own  typing  ability  in  terms  of 
speed  and  accuracy  respectively.  These  data  fields  are  also 
numeric  codes  for  the  response  given  on  the  questionnaire. 
A  value  of  1  represents  a  poor  rating,  a  2  an  average  rating 
while  a  3  represents  a  very  good  rating  in  the  speed  and 
accuracy  capability  of  each  subject.  Column  16  is  the 
response  to  the  question  of  whether  the  subject  ever  had  a 
typing  course.  The  same  response  convention  was  used  for 
the  two  previous  yes-no  questions. 

The  second  to  the  last  column,  column  17,  tabulates  the 
data  collected  representing  the  geographic  region  where  the 
subject  grew  up.  The  codes  have  the  following  meaning. 

1  -  South 

2  -  East 

3  -  Midwest 

4  -  Foreign 

5  -  West 

6  -  All  over  or  not  specified 

Finally  the  last  column,  column  18,  is  the  subject's 
number  of  years  experience  with  computers  and  computer 
terminals. 

As  it  was  already  mentioned  Appendix  F  contains  the 
ranked  data.  The  data  are  organized  in  the  same  manner  as 
previously  described  for  the  raw  data  except  some  of  the 
columns  are  the  ranks  of  the  data  collected.  There  were 
numerous  ties  and  the  rank  value  assigned  was  the  average  of 
the  ranks  that  would  have  been  assigned  to  them  had  '•here 
been  no  ties.  All  of  the  recognition  error  counts  were 


ranked,  as  were  the  age  data,  total  acceptability  rating 
data,  and  the  subject  experience  data. 

The  first  set  of  analysis  used  Spearman's  rank 
correlation  coefficient  (Conover,  1980)  as  a  test  statistic 
to  determine  if  there  was  any  negative  correlation  between 
the  number  of  errors  presented  to  a  subject  and  that 
subject's  acceptability  rating.  It  had  been  hypothesized 
that  as  the  number  of  errors  increased  the  user 
acceptability  would  decrease.  Therefore,  the  null 
hypothesis  was  that  the  number  of  errors  and  user 
acceptability  are  mutually  independent  or  had  no 
correlation.  The  alternative  hypothesis  was  that  these  two 
variables  are  negatively  correlated.  The  following 
tabulates  Spearman's  correlation  coefficient  calculated  on 
the  ranked  data  between  these  two  variables. 


all  errors 

-.250 

total  misr ecogni tions 

-.249 

non  recognitions 

-.250 

sound  alikes 

-.215 

random  misrecogn itions 

-.201 

At  a  significance  level  of  .02  5  the  null  hypothesis  can  be 
rejected  for  the  first  three  values.  It  appears  as  if  the 
desired  negative  correlation  is  present,  but  the  correlation 
is  very  slight  as  indicated  by  the  value  of  the  correlation 
coefficient.  The  hypothesis  of  mutual  independence  can  not 
be  rejected  between  the  sound  alike  errors  and  user 
acceptability,  and  the  random  errors  and  the  acceptability 
variable. 

Figure  1  is  a  graph  of  the  numeric  user  acceptability 
totals  versus  total  user  perceived  errors.  It  shows  that 
although  there  is  evidence  of  a  negative  correlation  there 
is  little  information  existing  as  to  where  or  at  what  error 
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level  the  acceptability  values  start  to  decrease.  In  fact 
the  real  problem  is  exemplified  by  observing  the  average 
response  values  for  various  groups  of  subjects.  Table  1 
contains,  for  various  groups  of  subjects,  the  averages  of 
the  total  values  for  the  three  acceptability  questions  along 
with  the  standard  deviation.  There  is  very  little 
difference  between  the  groups.  Even  the  group  that  observed 
■ore  than  a  15*  error  rate  still  rated  the  system  in  the 
"like  it"  and  "would  use  it"  category.  A  Kruskal-Wallis  test 
was  done  to  determine  if  these  groups  of  subjects  had  the 
identical  mean  response  values.  This  hypothesis  could  not 
be  rejected  at  the  .05  or  .1  significance  level.  The  test 
statistic  value,  T,  was  calculated  at  6.78  and  the  chi- 
square  distribution  quantile  for  the  four  degrees  of  freedom 
at  the  .05  level  is  9.488  and  at  the  .1  level  is  7.779. 
Therefore,  even  though  a  small  negative  correlation  is 
detected  in  the  data,  very  little  information  can  be  gained 
as  to  where  a  distinct  drop  occurs  in  user  acceptability 
values. 

Two  other  Spearman's  Rho  correlation  coefficient  tests 
were  done.  First  a  correlation  possibility  was  investigated 
between  the  ranked  values  of  age  and  the  ranked  values  of 
acceptability.  The  hypothesis  that  age  and  acceptability 
values  were  not  correlated  could  not  be  rejected. 
Spearman's  correlation  coefficient  was  calculated  at  .03. 
The  same  test  was  done  to  check  the  data  for  mutual 
independence  among  the  ranked  values  for  years  of  experience 
and  the  ranked  values  of  the  acceptability  totals.  Again 
the  independence  hypothesis  could  not  be  relected  with  a 
correlation  coefficient  of  -.02. 

In  addition  to  the  Krus kal-Wallis  test  previously 
mentioned,  a  series  of  similar  tests  were  completed  in  order 
to  determine  if  there  were  any  differences  in  the  mean 
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acceptability  responses  among  different  groups  of 
individuals.  Table  2  summarizes  these  tests.  None  of  the 
hypotheses  that  all  the  mean  acceptability  responses  were 
idenitical  could  be  rejected.  In  other  words  no  statisical 
differences  could  be  found  among  the  various  groups  tested. 

The  only  test  which  suggested  a  possible  difference  was 
the  test  between  the  groups  divided  by  geography.  This  was 
interesting  because  numerous  peo> la  approached  the 
experiment  asking  for  example,  if  this  machine  understood 
"Louisianian".  They  knew  they  had  a  distinct  southern 
accent,  and  if  it  recognized  them  they  were  very  surprised. 
This  could  account  for  the  relatively  high  Kruskal- Wallis 
statistic  even  though  the  groups  could  not  statistically  be 
shown  to  have  different  means. 

In  conclusion,  there  is  really  very  little  information 
present  in  the  data  collected.  For  this  reason  the 
experiment  was  cut  short  before  the  time  was  spent  at 
business  establishments.  It  is  hypothesized  that  there  are 
at  least  two  basic  flaws  in  the  experimental  design.  First 
the  experiment  was  working  with  an  advanced  technology. 
Some  of  the  people  who  were  tested  were  not  aware  that  voice 
recognition  existed.  In  fact,  numerous  people  answered  or. 
the  questionnaire  that  they  had  seen  voice  recognition  to 
computers  on  "Star-Trek".  With  an  at^i^ude  like  that  any 
recognition  capability  was  impressive.  If  this  is  true  it 
would  be  expected  that  there  might  have  been  a  larger 
difference  between  the  group  acceptability  averages  between 
those  who  had  seen  voice  recognition  equipment  before  and 
those  who  had  not.  Since  this  did  not  occur  we  can  only 
assume  that  the  group  which  had  seen  voice  recognition 
before  knew  about  its  user  dapendence  limitations  and  were 
equally  impressed  with  the  user  independence  capabilities 
being  demonstrated. 


The  second  problem  area  involves  the  lack  of  a  task  which 
needed  to  be  accomplished  within  the  experimental  framework. 
Reading  a  list  of  words  and  pointing  out  the  errors  did  not 
create  the  frustrating  situations  which  are  going  to  exist 
when  you  encounter  a  recognition  error  while  trying  to 
acccmplish  a  task.  Although  it  was  hoped  that  stopping  the 
experiment  each  time  an  error  occurred  would  provide  this 
feeling,  it  did  not  totally  simulate  the  frustration 
associated  with  task  completion.  Furthermore,  subjects  who 
had  never  seen  or  used  voice  recognition  equipment  had  a 
difficult  time  visualizing  how  the  equipment  would  actually 
be  used.  Although  an  explanation  was  given  to  each  subject 
at  the  beginning  of  the  experiment,  it  appears  as  if  the 
concept  was  not  totally  understood  by  everyone.  This  was 
evidenced  by  some  of  the  questions  asked  by  the  subjects 
after  the  experiment  was  finished.  If  there  had  been  a 
realistic  job  or  goal,  this  problem  could  be  alleviated. 


21 


GRRPH  OF  RRW  DRTR 


^ioure  1 

Graph  of  ?av:  Data 

7.2 


7.5  10.0  12.5  15.0  17.5 

PERCENTAGE  ERROR  RATE  OBSERVED 


%  Error  Rate 
Groups 

#  of  subjects 
in  group 

Average 

Standard 

Deviation 

0.0  -  1 . S 

19 

4.40 

4.46 

2.0  -  5.0 

13 

4.41 

4.49 

5.5  -  10.0 

23 

3.96 

4.08 

10.5  -  14.5 

16 

3.71 

3.93 

15.0  -  21.0 

7 

3.90 

4. 02 

_ 

TABLE  1. 

Acceptability  Response  Summary 
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GROUP 


Seen  Voice  equipment- 
not  seen  voice  equipment 


Used  Voice  equipment- 
not  used  voice  equipment 


Slow,  intermediate,  fast 
speed  typist 


Poor,  fair,  very  good 
typing  accuracy 


Typing  course- 
no  typing  course 


Part  of  country  raised 
i  n 


TEST  STATISTIC 


CHI-SQUARE  =  .05 


1.18 


3.65 


2.14 


3  .  84 


3.  84 


5.99 


4 . 38 


0.12 


5 . 99 


3. 84 


10.15 


11.07 


TABLE  2. 

Results  of  Krus kal - Wa 1 1 i s  Tests 
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?.  RECOMMENDATIONS  AND  CONCLUSIONS 


The  data  collected  in  this  experiment  do  not  help  answer 
the  question  of  how  good  a  voice  recognition  system  must  be 
to  maintain  user  acceptability.  The  following 
recommendations  are  provided  for  future  research  efforts  and 
experiments. 

1.  Introduce  the  subjects  to  voice  recognition 
equipment  before  the  start  of  the  experiment. 

2.  Demonstrate  the  equipment  within  an  appropriate 
work  environment.  In  other  words,  demonstrate  +he 
use  of  the  equipment  to  accomplish  a  suitable  job 
within  their  individual  areas  of  expertise. 

3.  Have  the  subjects  train  the  eguipmer.t.  Don’*- 
present  a  system  which  is  obviously  years  ahead  of 
currently  available  technology. 

H.  Set  up  a  series  of  tasks  which  are  suitable  for 
voice  recognition  input  which  can  be  accomplished 
within  the  test  environment.  It  appears  that  it 
is  important  that  the  job  is  very  realistic  in 
nature. 

5.  Consider  the  following  method  to  vary  the 
recognition  accuracy  for  the  experiment.  Use  a 
recognition  system  which  has  the  capability  to 
easily  access  not  only  the  word  which  was 
recognized,  but  the  runner  up  word.  Then  by  the 
use  of  a  random  process  determine  whether  the 
recognition  unit  should  output  the  first  or  runner 
up  word.  If  the  recognition  system  used  has  a 
fairly  good  accuracy  rate,  the  the  first  choice 
word  should  be  the  correct  word.  Therefore,  by 
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randomly  selecting  the  second  word  you  are 
randomly  selecting  errors  from  the  recognition 
unit.  It  is  possible  that  the  recognition  unit 
will  make  a  mistake  and  the  recognized  word  will 
be  incorrect.  If  the  random  draw  determines  a 
correct,  response  is  to  be  given,  the  first  word 
will  be  output  to  the  system  but  in  this  case  it 
is  an  error.  This  error  was  not  expected; 
therefore,  although  the  actual  accuracy  rate  of 
the  system  will  not  be  under  the  experimenters 
control  the  overall  error  rate  should  be  very 
close  to  the  percentage  of  times  the  runner  up 
word  is  chosen.  If  a  100%  accurate  system  was 
used  this  percentage  of  runner  up  choices  wculd  be 
equal  to  the  error  rate  observed.  Since  a  100% 
accurate  system  does  not  exist  it  appears  as  if 
close  will  have  to  do. 

The  Interstate  Electronics  Corporation  machine  is 
suitable  for  this  type  of  task.  The  only  problem  involved 
with  this  recommendation  is  that  it  will  be  impossible  to 
observe  error  rates  much  less  than  the  underlying  error  rate 
associated  with  the  equipment  chosen  for  use  in  *he 
experiment. 

Since  there  was  evidence  of  the  hypothesized  negative 
correlation,  it  is  possible  that  the  frustration  measurement 
will  fulfill  the  needs  of  follow  on  experiments.  This 
question  of  frustration  measurement  should  be  investigated 
further  before  undertaking  the  next  phase  of  experimentation 
to  answer  the  question  about  acceptable  accuracy  rates  for 
voice  recognition  equipment. 
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APPENDIX  A 


♦LIST  1.9000 


I  10  Dt  *  ""!  REM  Dt  CONTAINS  A  CNTRL  D  | 

20  I*  -  '"«  REM  It  CONTAINS  A  CNTRL  T 

30  DIM  Ft(3) : Ft ( 1 )  =  “COMPUTER" :Ft(2)  *  "STOCK" : Ft<3)  *  "AIRLINE" 

60  DIM  W$(200. 2) 

50  PRINT  "DO  YOU  WANT  TOT"!  PRINT  "1.  CREATE  H  UEU  FILE":  TRINT  "2.  UPDAT 
E  A  FILE":  PRINT  "3.  PRINTOUT  A  FILE":  PRINT  "6.  STOP"  j 

£0  INPUT  A  | 

70  ON  A  GOTO  1000, '2000,  300074000 
1000  Bt  =  "CREATE" 

1010  GOSUB  8000 

1020  PRINT  Dt ;  "OPEN  "  fPt(A)  ! " .  V00ir:  PRINT  Dtl^CLOSE" 

1030  I  *  i:J  -  1 

_ 1040 _  GOSUB  5000  _ 

1050  GOTO  50 
2000  Bt  *  "UPDATE" 

2010  GOSUB  8000 

2020  ONERR  GOTO  2060 

2030  PRINT  Dt ; "OPEN  " ;Ft(A) » V001 “ 

2040  PRINT  Dt ? “READ  "  ;  F  t  <  A ) 

— 2050  FOR  I TO  200*  FE)R  J  “  f  TIT  2's  “INPUT  WtdTJ  I'S  NEXTTr- NEXT  I - 

2060  PRINT  Dt; “CLOSE" 

2070  PRINT  "DO  YOU  WANT  TO:":  PRINT  "1.  CHANGE  INDIVIDUAL  WORDS":  PRINT  " 


2080 

2090 

2100 


2.  ADD  TO  THE  LIST  OF  WORDS" :  PRINT  "3.  STOP' 
INPUT  AA 

ON  AA  GOTO  2100.2200.2300 


PRINT  "WHAT  WORD  NUMBER 
0  STOP  INPUT  A  -1" 

2110  INPUT  I:  IF  I  (0  THEN  GOTO  2170 

PRI NT _ " DO  YOlTWA NT~T 0  C HANOE11^  PRTNT  "  IV  THE~WDRD"  T  PRINT' 


2120 


2130 


rZ7~THES0 


UND  ALIKE" 

_ INPUT  J 

PRINT 


2140  PRINT  I.J 
2150  INPUT  Wt(I.J) 
2160  GOTO  2100 


2170 

2180 

2200 

I  *  20 1:  GOSUB  O07CT” 

GOTO  2070 

PRINT  "WHAT  WORD  NUMBER  DO  YOU  WANT  TO  START 

AT?" 

2210 

INPUT  I : J  =  1 

2220 

GOSUB  5000 

2230 

GOTO  2070 

2300 

GOTO  50 

3000 

Bt  =  "PRINTOUT" 

3010 

GOSUB  6000 

3020 

ONERR  GOTO  3060T~ 

3030 

PRINT  Dt * “OPEN  " *Ft( A) » " . V001 " 

3040 

PRINT  Dt ? " READ  " ;Ft< A) 

3050 

FOR  I  =  1  TO  200:  FOT*T~  =  I  1U  2:  InhUJ  "RtCTT 

~J7 !  NEXT  JT  NEXT  I 

3060 

PRINT  Dt; "CLOSE" 

3070 

PRINT  "DO  YOU  WANT":  PRINT  "1.  JUST  THE  WORD 

LIST":  PRINT  "2.  WORD 

1ST  AND  SOUND  ALlKES'r:  PRIN’T_n37  5T0P"~~ 

3080 

INPUT  J 

3085 

ON  J  GOTO  3090,  3090, 3400 
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S&90  PRINT  *‘00  YOU  NAVE' OKr=TMflGE"XORDE9  flNa~H»NT-fl  HARDCOPY?  CY  OR  N>". 
3100  INPUT  Y* 

3110  IF  Y*  *  “N"  THEN  GOTO  3170 

3120  PRINT  D*;"PR#'2,r  ' 

3130  PRINT  I*. "S0N" 

3170  ON  J  GOTO  3200.3300.3400 

— 3200  'FOR'”T~5‘~l~TO ‘200 :  PRINT"  r,  W*a,J)  r  NEXT  I - 1 

3210  PRINT  D*; "PR#  0":  GOTO  3070  j 

3300  FOR  I  *  1  TO  200:  PRINT  I.WSCI.J  -  1).W*(I.J>:  NEXT  I  [ 

33i0  PRTNfH D*;  "PR#  O'"!  GOTO- 3070 
3400  GOTO  50 
4000  STOP  :  END 

5000“  IF  fT~200  THEN  GOTO  50E5 
5010  PRINT  I.J 

_ 5020  INPUT  N»<I,J) _ _ _ 

5030  iF“W*<I,  j)  =  "END"  THEN  GOTO  5065 
5040  J  =  J  +  1 

_ 5050  IF  J  3  2  THEN  GOTO  501 0 _ 

5060  I  ■  I  +  1-*J  =  l:  GOTO  5000 

5065  GOSUB  5070:  RETURN 

5070  PRINT  D*.  "OPEN  “  *,F$<  A)  5  ■ .  V001 " 

5080  PRINT  Dif "WRITE  "!F*(A) 

5090  FOR  K  =  1  TO  I  -  l:  FOR  J  -  1  TO  2:  PRINT  W*CK. J) :  NEXT  J:  NEXT  K 
5100  PRINT  D*? "CLOSE":  RETURN 

6000  PRINT-'' WHAT~FTLE_DCrYOD  WANT  TO  PRINT  "  IT  "COMPUTER" :~PRINT  "2* 

STOCK":  PRINT  "3.  AIRLINE"  j: 

6010  INPUT  A:  RETURN _  1| 


Airline  Vocabulary 


1 .  All entown 

2.  Abilene 

3.  Albuquerque 

4.  East  Hampton 

5.  Washington  Dulles 

6.  Columbus  Ohio 

7.  Akron 

8.  Ohara  Field 

9.  Cleveland 

10.  Cedar  Rapids 

11.  Corpus  Christi 

12.  San  Juan 

13.  Boston 

14.  Rapid  City 

15.  Daytona  Beach 

16.  Goose  Bay 

17.  Dayton 

18.  Utica 

19.  Denver 

20.  Lewiston 

21.  Detroit 

22.  Montpelier 

23.  Charleston 

24.  Monterey 

25.  Franklin 

26.  Bangor 

27.  Fargo 

28.  Manitowoc 

29.  New  York 

30.  San  Jose 

31.  Fort  Collins 

32.  Laramie 

33.  Kona 

34.  San  Diego 

35.  Freeport 

36.  Missoula 

37.  Oklahoma  City 

38.  Colorado  Springs 

39.  Pittsburgh 

40.  Duluth 

41.  Lousville 

42.  St  Petersburg 

43.  Honolulu 

44.  Kirksville 

45.  Columbia 

46.  Columbus  Georgia 

47.  Roanoke 

48.  Rockford 

49.  Washington  National 

50.  Jacksonville 


Morgantown 

Aberdeen 

Kodiak 

Birmingham 

Washington  National 

Columbus,  Georgia 

Charleston 

Omaha 

Lakeland 

Grand  Rapids 

Cincinnati 

South  Bend 

Boise 

Sioux  City 

Long  Beach 

Pine  Coy 

Galveston 

Sitka 

Beaver 

A1 1 entown 

Duluth 

Eau  Claire 

Kinston 

San  Jose 

Richland 

Boston 

Elko 

Roanoke 

Norfolk 

San  Diego 

Ft  Smith 

Miami 

Helo 

San  Francisco 

Bridgeport 

Honolulu 

Jefferson  City 

Hot  Springs 

Plattsburgh 

Detroit 

Knoxville 

Rochford 

Missoula 

Louisville 

At! anta 

Columbus,  Ohio 

Manitowoc 

Medford 

Washington  Dulles 
Jackson 
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51 .  Atlantic  City 

52.  Chico 

53.  Araaha 

54.  Plattsburgh 

55.  Eau  Claire 

56.  Norfolk 

57.  Cheyenne 

58.  San  Jose 

59.  Phell ipsburgh 

60.  Helo 

61 .  Medford 

62.  Beaver 

63.  Lewis  town 

64.  Sitka 

65.  Galveston 

66.  Grand  Rapids 

67.  Dubuque 

68.  Pine  Coy 

69.  Richland 

70.  Long  Beach 

71 .  South  Bend 

72.  Birmingham 

73.  Sioux  City 

74.  Cincinnati 

75.  Kenston 

76.  Acapulco 

77 .  Nome 

78.  Orlando 

79.  Pompano  Beach 

80.  Harrisburg 

81.  Hot  Springs 

82.  Pittsfield  , 

83.  Kodiak 

84.  Milwaukee 

85.  Knoxvill 

86.  Ft.  Smith 

87.  Paris 

88.  Morgantown 

89.  Springfield 

90.  El ko 

91 .  New  Bedford 

92.  West  Palm  Beach 

93.  Bridgeport 

94.  Kokomo 

95.  Aberdeen 

96.  Jackson 

97 .  MenomTmee 

98.  Fort  Wayne 

99 .  Rome 

100.  Toledo 

101.  Rochester 

102.  Kalamozoo 

103.  Providence 


Oklahmoa  City 

Toledo 

OHara  Field 

Pittsburgh 

Montpelier 

New  York 

San  Juan 

Monterey 

St.  Petersburg 

Kona 

New  Bedford 

Denver 

Lewiston 

Utica 

Dayton 

Cedar  Rapids 

Albuquerque 

Goose  Bay 

Cleveland 

Daytona  Beach 

Cheyenne 

East  Hampton 

Rapid  City 

Corpus  Christi 

Akron 

Chico 

Rome 

Kokoma 

West  Palm  Beach 
Paris 

Colorado  Springs 

Springfield 

Dubuque 

Menominee 

Kirksville 

Fort  Collins 

Harrisburg 

Lewis town 

Pitts  field 

Fargo 

Phillipsburg 

Pompana  Beach 

Freeport 

Orlando 

Abilene 

Jacksonville 

Milwaukee 

Fort  Lauderdale 

Nome 

Acopulco 

Worchester 

Waterloo 

Provincetown 
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104.  Morristown 

105.  Clarksburg 

106.  Tallahassee 

107.  San  Angelo 

108.  Martha  Vineyard 

109.  Gainesville 

110.  Sacremento 

111.  Lake  Charles 

112.  Osage  Beach 

113.  Brookings 

114.  Pueblo 

115.  Antiguo 

116.  Knoxville 

117.  Larado 

118.  Ogdensburg 

119.  Sumter 

120.  Show  Low 

121.  Wichita  Falls 

122.  Worthington 

123.  Provincetown 

124.  Waterloo 

125.  Apple  Valley 

126.  Appleton 

127.  Modesto 

128.  Worchester 

129.  Huntsville 

130.  Waterville 

131.  Baton  Rouge 

1 32 .  Marquette 

133.  New  Orleans 

134.  Walla  Walla 

135.  Tupelo 

136.  Astoria 

137.  Catskills 

138.  Atlanta 

139.  Janesville 

140.  Durango 

141.  Newburg 

142.  Trenton 

143.  Billings 

144.  Fayetteville 

145.  Greenvill 

146.  Buffalo 

147.  Evansville 

148.  Phoenix 

149.  Greensboro 

150.  Grand  Junction 

151.  Anchorage 

152.  Wichita  Falls 

153.  San  Francisco 

154.  Waterville 

155.  San  Juan 

156.  Princeton 


Worthington 

Marthas  Vineyard 

Apple  Valley 

Show  Low 

Ogdensburg 

Knoxville 

Larado 

Lake  Tahoe 

Anchorage 

New  Orleans 

San  Angelo 

Altoona 

Waterville 

El  Dorado 

Lynchburg 

Rochester 

Tampico 

Wichita 

Riverton 

Providence 

Kalamazoo 

Tallahassee 

Trenton 

Durango 

Sumter 

Catskills 

Janesville 

Osage  Beach 

Phoenix 

Billings 

Antingua 

Modesto 

Augusta 

Greinville 

Bermuda 

Huntsville 

Buffalo 

Clarksburg 

Grand  Junction 

Brookings 

Gainsesville 

Evansville 

Greensboro 

Jacksonville 

Marquette 

Ontario 

Morristown 

Baton  Rouge 

Twin  Falls 

San  Juan 

Greenville 

San  Jose 

Pendleton 


32 


157. 

Churchill  Falls 

158. 

Hastings 

159. 

Wichita 

160. 

Fort  Lauderdale 

161. 

Pueblo 

162. 

Saint  Louis 

163. 

El  Dorado 

164. 

Grand  Cayman 

165. 

Jefferson  City 

166. 

Idaho  Falls 

167. 

Pendleton 

168. 

Greenville 

169. 

Miami 

170. 

Grand  Canyon 

171. 

St  Croix 

172. 

Lakeland 

173. 

Harrison 

174. 

Boise 

175. 

Twin  Falls 

176. 

Riverton 

177. 

Ontario 

178. 

Lynchburg 

179. 

Altoona 

180. 

Agusta 

181  . 

Victoria 

182. 

Sarosota 

183. 

Bermuda 

184. 

Poplar  Bluff 

185. 

Tulsa 

186. 

Jacksonville 

187. 

Poza  Rica 

188. 

Guadalahara 

189. 

Pensacola 

190. 

Orlando 

191. 

Topeka 

192. 

Wausau 

193. 

Galesburg 

194. 

Penta  Gorda 

195. 

Lake  Tahoe 

196. 

Danville 

197. 

Mankato 

198. 

Hattiesburg 

199. 

Lampico 

200. 

Clinton 

Idaho  Falls 
Harrison 
Wichita  Falls 
Fort  Wayne 
Sacramento 
St.  Croix 
Pueblo 

Grand  Canyon 
Atlantic  City 
Churchill  Falls 
Princeton 
Waterville 
Laramie 
Grand  Cayman 
Saint  Louis 
Frankl in 
Hastings 
Bangor 

Wichita  Falls 

Cl  inton 

Orlando 

Galesburg 

Astoria 

Wausau 

Topeka 

Pensacola 

Columbia 

Council  Bluffs 

Guadalahara 

Danville 

Walla  Walla 

Sarasota 

Poza  Rica 

Mankato 

Tillsa 

Victoria 

Newburg 

Tupelo 

Lake  Charles 

Fayetteville 

Pueblo 

Clarksburg 

Punta  Gorda 

Appleton 


Computer  vocabulary 


1  Login 

2  A1 ter 

3  Backward 

4  Conmand 

5  Cursor  Column 

6  Findup 

7  Left 

8  Nfind 

9  Put-D 

10  Query 

11  Reset 

12  Set  Autosave 

13  Set  File  Mode 

14  Set  Line  Character  Off 

15  Set  Number 

16  Set  Reserved 

1 7  Set  Synonym 

18  Set  Verify 

19  Stack 

20  Delete  Line 

21  Assign 

22  Five 

23  Six 

24  Seven 

25  Fight 

26  Nine 

27  Ten 

28  Alpha 

29  Bravo 

30  Charlie 

31  Logoff 

32  Bottom 

33  Cmsg 

34  C-Repalce 

35  Find 

36  Join  Cursor 

37  Msg 

38  Next 

39  Purge 

40  Replace 

41  Set  Arbitrary  Character 

42  Set  Filler 

43  Set  Line  Charac  >r  On 

44  Set  Nulls 

45  Set  Record  Format 

46  Set  Stream 

47  Set  Variable  Blank 

48  Split 

49  Uppercase 

50  Duplicate  Line 

51  Access 

52  Filedef 

53  Global 

54  Listfile 

55  Start 


Logoff 

Add 

Bottom 
Compress 
Cursor  File 
Forward 
Load 

Nfind-up 

Parse 

Quit 

Restore 

Set  Case 

Set  File  Name 

Set  Logical  Record  Length 

Set  Pack 

Set  Scale 

Set  Tableine 

Set  Wrap 

Status 

Duplicate  Line 

Access 

Four 

Sixty 

Seventy 

Eighty 

Ninety 

Send 

Papa 

Romeo 

Whiskey 

Login 

Backward 

Command 

Cursor  Column 

Findup 

Join 

Macro 

Nfind 

Put-D 

Reset 

Set  Autosave 

Set  File  Mode 

Set  Line  Character  Off 

Set  Number 

Set  Reserved 

Set  Synonym 

Set  Verify 

Stack 

Up 

Delete  Line 

Assign 

Fetch 

Cobol 

Filedef 

State 
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56 

57 

58 

59 

60 
61 
62 

63 

64 

65 

66 

67 

68 

69 

70 

71 

72 

73 

74 

75 

76 

77 

78 

79 

80 
81 
82 

83 

84 

85 

86 

87 

88 

89 

90 

91 

92 

93 

94 

95 

96 

97 

98 

99 
100 
101 
102 

103 

104 

105 

106 

107 

108 

109 

110 


Zero 

One 

Two 

Three 

Four 

Profile 

Cancel 

Cms 

Cp 

Expand 

File 

Join  Column 

Modify 

Power- Input 

Renumber 

Selective  Change 

Set  Current  Line 

Set  Image 

Set  Message  Mode 

Set  Prefix 

Set  Span 

Set  Text 

Sort 

Type 

Debug 

Papa 

Quebec 

Romeo 

Sierra 

Tango 

Uni  form 

Victor 

Whiskey 

Yankee 

Zulu 

No  Profile 

C-Append 

C-Locate 

C-overlay 

Dupl icate 

Emsg 

Put 

Join 

Move 

Preserve 

Repeat 

Set  APL 

Set  Escape 

Set  Implicitly  Cms 

Set  Nondisplayable  Characters 

Set  Range 

Set  Stay 

Set  Truncate  Column 

Sos 

Up 


Kilo 

Echo 

Echo 

Thirty 

Five 

No  Profile 

C-Append 

Cmsg 

C-Replace 

Emsg 

Find 

Join  Cursor 
Move 

Preserve 
Repeat 
Set  APL 
Set  Escape 

Set  Implicityly  to  CMS 
Set  Nondisplayable  Characters 
Set  Range 
Set  Stay 

Set  Truncate  Column 

Sos 

Up 

Disk 

Alpha 

Stack 

Tango 

A1  pha 

Romeo 

Nine 

Preserve 

X-ray 

Charlie 

Move 

Profile 

Cancel 

Cms 

Cp 

Down 

Expand 

Input 

Join  Column 
Msg 
Purge 
Replace 

Set  Arbitrary  Character 

Seet  Filler 

Set  Line  Character  On 

Set  Nulls 

Set  Record  Format 

Set  Stream 

Set  Variable  Blank 

Split 

Uppercase 
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m 

Disk 

Debug 

112 

X-Ray 

Yankee 

113 

Twenty 

Ninety 

114 

Thirty 

Eighty 

115 

Forty 

Seventy 

116 

Fifty 

Five 

117 

Sixty 

Fi  fty 

118 

Seventy 

Seven 

119 

Eighty 

Thirty 

120 

Ninety 

Twenty 

121 

Xedit 

Update 

122 

C-Delete 

C-First 

123 

C-Last 

C-Locate 

124 

Compress 

Copy 

125 

Cursor  File 

Cursor  Screen 

126 

Forward 

Find-up 

127 

Get 

Set 

128 

Load 

Locate 

129 

Nfind-up 

Next 

130 

Overlay 

On 

131 

Quit 

Q- Qu i t 

132 

Restore 

Right 

133 

Set  Case 

Set  Command  Line 

134 

Set  File  Name 

Set  File  Type 

135 

Set  Logical  Word  Length 

Set  Macro 

136 

Set  Pack 

Set  Program  Functic 

137 

Set  Scale 

Set  Screen 

138 

Set  Tabline 

Set  Tabls 

139 

Set  Wrap 

Set  Zone 

140 

Status 

Stack 

141 

Top 

Transfer 

142 

Compare 

Cp 

143 

Erase 

Disk 

144 

Fetch 

Fi ledef 

145 

State 

Start 

146 

Delta 

India 

147 

Echo 

One 

148 

Foxtrot 

Findup 

149 

Gulf 

Off 

150 

Hotel 

Hel  p 

151 

Update 

Xedit 

152 

C- First 

Change 

153 

C-Insert 

C-Last 

154 

Copy 

Count 

155 

Cursor  Screen 

Cursor  File 

156 

Delete 

Down 

157 

Set 

Get 

158 

Help 

Hextype 

159 

Locate 

Lowercase 

160 

On 

Off 

161 

Q-Quit 

Query 

162 

Read 

Recover 

163 

Right 

Read 

164 

Set  Command  Line 

Set  Column  Pointer 

165 

Set  File  Type 

Set  Hexidecimal 

Key 
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166 

Set  Macro 

Set  Mask 

167 

Set  Program  Function  Key 

Set  Point 

168 

Set  Screen 

Set  Serial 

169 

Set  Tabls 

Set  Terminal 

170 

Set  Zone 

Set  Equals 

171 

Transfer 

Type 

172 

CP 

Compare 

173 

Cobol 

Global 

174 

India 

Lima 

175 

Jul iett 

Delete 

176 

Kilo 

Zero 

177 

Lima 

India 

178 

Mike 

Quit 

179 

November 

Oscar 

180 

Oscar 

A1  ter 

181 

Add 

Alter 

182 

Change 

C-Insert 

183 

Count 

C-Overlay 

184 

Down 

Duplicate 

185 

Hextype 

Help 

186 

Input 

Put 

187 

Lowercase 

Left 

188 

Macro 

Modi fy 

189 

Off 

Overlay 

190 

Parse 

Power- Input 

191 

Recover 

Renumber 

192 

Save 

Selective  Change 

193 

Set  Column  Pointer 

Set  Current  Line 

194 

Set  Hexidecimal 

Set  Image 

195 

Set  Mask 

Set  Message  Mode 

196 

Set  Point 

Set  Prefix 

197 

Set  Serial 

Set  Span 

198 

Set  Terminal 

Set  Text 

199 

Set  Equals 

Set  Prefix 

200 

Shift 

Sort 
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Stock  vocabulary 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

49 

50 

51 

52 

53 

54 

55 


Houston  Natural  Gas 

IBM 

ITT 

Iowa  Electric 
K-Mart 

Kaiser  Aluminum 
Lenox 

Leer-Siegler 

Lionel 

Iowa  Gas 

Litton 

Kaiser  Steel 

Lockheed 

MGM 

Magic  Chef 

Marriott 

Marathan  Oil 

Mattel 

Maytag 

Memorex 

McDonalds 

AMF 

McDonald-Douglas 

3M 

NCR 

Nabisco 
Natomas 
Oklahoma  Gas 
Penney* s 

Combustion  Engineering 

Can  Edeson 

Dennys 

Macys 

Dow  Chemical 

Elgin 

Dorsey 

Florida  Power  &  Light 
Greyhound 
Hal i burton 
Kroehler 

Wisconsin  Power  &  Light 

Grumman 

U.S.  Air 

U.S.  Steel 

U.S.  Life 

Western  Union 

Zapata 

Air  Florida 

Wennebago 

Air  Wisconsin 

Atlantic  Richfield 

BDM 

Tampox 

Alabama  Power 


Oklahoma  Gas 

MGM 

IBM 

Iowa  Gas 

Gillette 

Kaiser  Steel 

Purex 

Singer 

Mattel 

Iowa  Electric 
Hal iburton 
Kaiser  Aluminum 
Square  D 
IBM 

Zenith 
Mattel 
Ashland  Oil 
Marriott 
Rubber  Maid 
Tampox 

McDonald-Douglas 

ATT 

McDonalds 

BDM 

CBS 

Sambos 

Ameroda  Hess 
Houston  Natural  Gas 
Macy's 

Cummins  Engine 

Western  Union 

Dorsey 

Penneys 

Dow  Jones 

Hal iburton 

Dennys 

Wisconsin  Power  &  Light 

Grumman 

Elgin 

Burroughs 

Florida  Power  &  Light 

Greyhound 

U.S.  Gypsum 

U.S.  Air 

U.S.  Steel 

Con  Edison 

Aenith 

Air  Wisconsin 

Whirlpool 

Air  Florida 

Atlantir  Cesearch 

MGM 

Zerox 

Florida  Power  &  Light 
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56 

Bank  of  Virginia 

57 

Burroughs 

58 

CBS 

59 

Atlantic  Research 

60 

Zenith 

61 

Xerox 

62 

Bank  of  America 

63 

John  Deer 

64 

Delta  Airlines 

65 

Dow  Jones 

66 

Eastern  Airlines 

67 

General  Motors 

68 

General  Steel 

69 

General  Radio 

70 

Georgia  Pacific 

71 

Goodyear 

72 

Gillette 

73 

Gulf  Oil 

74 

Hicla  Mining 

75 

Goodrich 

76 

Heinz 

77 

Pei  rex 

78 

Sambos 

79 

Singer 

80 

Square  D 

81 

Collins  Radix) 

82 

Tandy 

83 

Texaco 

84 

United  Airlines 

85 

U.S.  Gypsum 

86 

Ala  Moana 

87 

Albertsons 

88 

Gulf  Western 

89 

Dome  Mining 

90 

Ameroda  Hess 

91 

American  Broadcasting 

92 

Honda 

93 

Allis  Chalmers 

94 

American  Hospital 

95 

Ashland  Oil 

96 

Bank  of  America 

97 

Bendix 

98 

Boise  Cascade 

99 

Canadian  Pacific 

1Q0 

Ben  Gay 

101 

Champion 

102 

Chrysler 

103 

Coca  Cola 

104 

Clorex 

105 

Chase  Manhattan 

106 

Data  General 

107 

Cities  Services 

108 

Canoca 

109 

Southern  Pacific 

110 

Boeing 

111 

Ford 

Bank  of  America 

Kroehler 

NCR 

Atlantic  Richfield 

Zapato 

Lenox 

Bank  of  Virginia 
Chrysler 
Eastern  Airlines 
Dow  Chemical 
Delta  Airlines 
General  Radio 
Data  General 
General  Motors 
Canadian  Pacific 
Goodrich 
K-Mart 

Gulf  Western 
Dome  Mining 
Goodyear 
Honda 
Clorox 

Collins  Radio 
Leer  Siegler 
Tandy 
Texaco 
Square  D 
Nabisco 

Eastern  Airlines 
U.S.  Life 
Coca  Cola 
Alles  Chalmers 
Gulf  Oil 
Hiclo  Mining 
Natomas 

American  Hospital 
Heinz 

A1 bertsons 

American  Broadcasting 
Marathan  Oil 
Bank  of  California 
Ben  Gay 
Ben  Gay 

Georgia  Pacific 
Bendex 

Chose  Manhatten 
John  Deer 
Ala  Moana 
Memo rex 
Champion 
General  Steel 
Citicorp 
Coca  Cola 
Georgia  Pacific 
Boise  Cascade 
Fotomat 


112 

113 

114 

115 

116 

117 

118 

119 

120 
121 
122 

123 

124 

125 

126 

127 

128 

129 

130 

131 

132 

133 

134 

135 

136 

137 

138 

139 

140 

141 

142 

143 

144 

145 

146 

147 

148 

149 

150 

151 

152 

153 

154 

155 

156 

157 

158 

159 

160 
161 
162 

163 

164 

165 


El  Paso  Gas 

Snion  Oil  of  California 
TRW 

Citicorp 
Revlon 
Woolworth 
Rockwell  . 

United  Technologies 

Fotomat 

San  Diego  Gas 

Southwestern  Gas 

Bell  &  Howell 

Honeywell 

Avon 

Threshold  Technology 
Allied  Corp. 

ACF  Industries 
Beckman  Instruments 
Big  Three 
Boston  Edison 
Brooklyn  Union  Gas 
Bui ova 

Campbell  Soup  .  .  ht 
Carolina  Power  &  Lijht 
Carpenter  Technology 
AMF 

Becton  Dickinson 
Carolina  Freight  Carriers 
Central  Maine  Power 
Clark  Oil  &  Refining 
Col  dwell  Banker 
Colonial  Store 

Commonwealth  Edison 

Central  &  South  West 
Consolidated  Foods 
Continental  Can 
Colt  Industries 
Clark  Equipment 
Central  Soya 
Continental  Oil 

SSSlES  Freeway 

Continental  Air  Lines 
Corning 

Cummins  Engine 
Cooper  Laboratories 
Cyprus  Mines 
Del  Monte  Corp 

Continental  Telephone 
Detroit  Edison 
Diebold 

Dresser  Industries 
Eastman  Kodak 
Crocker 


San  Diego  Gas 
Bank  of  California 
MCR 

Cities  Services 
Avon 

Whirlpool 
Honeywel 1 

Threshold  Technology 
Ford 

El  Paso  Gas 
San  Diego  Gas 
Rockwel 1 
Rockwel 1 
Revlon 

United  Technology 
Alles  Chalmers 
AMF  .  .  . 

Becton,  Dickinson 
Square  D 
Con  Edison 
San  Diego  Gas 

Burroughs 

Honeywel 1  .  , .nht 

Wisconsin  Power  &  Light 
United  Technology 
ACF  Industries 
Beckman  Instruments 
Carolina  Power  &  Light 
Central  &  South  West 
Clark  Equipment 
Rockwel 1 
Colt  Industries 
Con  Edison 

Consolidated  Freightway 
Continental  Air  Lines 
Colonial  Stores 
Hark  Oil  &  Refining 
Central  &  South  West 
Continental  Telephone 
Cooper  Laboratories 
Consolidated  Foods 
Continental  Can 

Combustion  Engineering 
Cooper  Range 

Bucyrus-Erie 

Delta  Air  Lines 
Continental  Oil 
Commonwealth  Edison 
01  Giorgio 
Drey f ess 

Eastern  Airlines 
Corning 
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166 

Bucyrus-Erie 

Cyprus  Mines 

167 

Emuson  Electric 

Emery  Air  Freight 

168 

Empire  Gas 

Emery  Air  Freight 

169 

Di  Georgia 

Georgia  Pacific 

170 

Dreyfus 

Dresser  Industries 

171 

Faberge 

Fairchild 

172 

Federal  Mogul 

Federal  Paper  Board 

173 

Fieldcrest 

First  Chicago 

174 

Fairchild 

Faberge 

175 

Emery  Air  Freight 

Emerson  Electric 

176 

Ford  Motor 

Foremost 

177 

General  Dynamics 

General  Food 

178 

Gerber 

Gillette 

179 

Grand  Union 

Estern  Union 

180 

Hushey 

Hewlett-Packard 

181 

Hilton 

Holiday  Inn 

182 

Host 

Houston  Natural  Gas 

183 

Inland  Steel 

Kaiser  Steel 

184 

Internation  Harvester 

International  Paper 

185 

Johnson  &  Johnson 

Jonathan  Logan 

1«6 

Kennecaft  Copper 

Kimberly  Clark 

187 

Lone  Star  Gas 

Lone  Star  Industries 

188 

Lukens  Steel 

Kaiser  Steel 

189 

First  Chicago 

Fieldcrest 

190 

General  Food 

General  Dynamics 

191 

Holiday  Inn 

Hilton 

192 

Foremost 

Ford  Motor 

193 

Hewlett  Packard 

■'ershey 

194 

Houston  Natural  Gas 

Host 

195 

International  Paper 

International  Harve 

196 

Johnathan  Logan 

Johnson  &  Johnson 

197 

Magnavox 

Lenox 

198 

Lone  Star  Industries 

Lone  Star  Gas 

199 

Federal  Paper  Board 

Federal  Mogul 

200 

Kimberly  Clark 

Kennecott  Copper 
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appendix  c 


'5 


_  -ttLL15I_l  jJ3008_ 


10  DIM  Wi<200,2),Fi<3),  P<4) 

20  D*  =  .  REM  CONTAINS  ft  CONTROL-D 

30  Fi(l)  =  "COMPUTER"  :  Fi  (2)  =  "STOCK  "  :  F  i <f 37  ~  =  11 A I RCTNET"" 
40  PCI)  =  . 99s  P<2)  =  . 95! P<3)  =  .90:P<4)  =  .85 
45  Mi  =  " 


50 

PRINT  "WHftT  WORD  DATABASE  DO  YOU  WANT  TO  USE?":  PRINT  T 

.  "COMPUTER" 

• 

• 

"2.  STOCK":  PRINT  "3.  AIRLINE" 

i 

£0 

INPUT  ft 

J 

70 

PRINT  Di;"OPEN  "jFi<ft) V00i" 

80 

PRINT  Di! " READ  ";F$(A> 

90 

FOR  I  *  1  TO  200:  FOR  J  =  1  TO  2:  INPUT  Wi(I,J>:  NEXT  J: 

NEXT  I 

100 

PRINT  Di i " CLOSE  " 

”1 

105 

PRINT  "INPUT  ft  SEED":  INPUT  IS 

i 

110 

PRINT  "WHftT  ALGORITHM  DO  YOU  WANT  TO  USE?" 

J 

_ j 

120 

Input  b.*  if  b  <  0“Thent  stop  - 

130 

GOSUB  40000 

140 

CALL  JTftBLE  +  15 

150 

VOCi  =  " CLASS! CTVOC"  '  ” - 

180 

GOSUB  40100 

I 

1 

170 

PRINT  "FIRST  SOME  PRACTICE  WORDS" 

! 

180 

1  =  1 

' 

190 

PRINT  I, Will, 1):  GOSUB  40400:  FDR  Z  =  1  TO  150:  NEXT  Z: 

GOSUB  300 

200 

IF  I  <11  THEN  GOTO  190 

210 

Q  ="0:5“  =  0 :  N  =  0 :  T"=  ~  0 

220 

PRINT  "WE  WILL  NOW  START  THE  EXPERIMENT" 

230 

1  =  1 

1 

\ 

240 

P R I NT  I,  W* <I.l)i  “GO'5UEr20400':~F 0 R  T  =  T"TU  150!  NEX1  Z: 

GOSUB  300 _ 

250 

IF  I  <201  THEN  GOTO  240 

280 

PRINT  "THftNK  YOU  FOR  PARTICIPATING  IN  THE  EXPERIMENT" 

270 

T  =  <200  -Q-S-NW.2  "  . . 

280 

PRINT  B;TJ 5Q; JS) ?N 

I 

< 

290 

GOTO  110 

J 

300 

IF  RND  CIS)  >  P(B)  THEN  GOTO '320 

310 

PRINT  M»;W$< I . 1 ) :  GOTO  400 

320 

R  =  RND  <IS> 

330 

IF  R  <  .33  THEN  GOTO- 390 

1 

340 

IF  R  <  .66  THEN  GOTO  360 

. 

350 

PRINT  M$J"?" :Q  =  Q  +  1 :  GOTO  400 

f 

380 

U  =  I N'T  ('  RND  <I5)  *  200T_+  "1 - — - — 

370 

IF  U  =  I  THEN  GOTO  360 

380 

PRINT  M$;W$(U, 1):N  =  N  +  It  GOTO  400 

390 

PRINT  Mi  J  WiTI ,  2 ) :  S  ='5+1  1 - 

400 

FOP  Z  =  1  TO  150:  NEXT  Z:I  =  I  +  1!  PRINT  :  RETURN 

\ 

f 

~BTT 


Appendix  D 


Code  # 


CATEGORY 

[  3 

AIRLINE 

[  3 

STOCK  BROKER 

[  3 

COMPUTER 

Age 


TIME  SPENT  AT  A  COMPUTER  TERMINAL  A  DAY 


hrs . 


1.  Have  you  used  voice  recognition  equipment  before 

2.  Have  you  seen  voice  recognition  equipment  used  before 


do  you  like  the  voice  system. 


□ 

YES 

□ 

NO 

□ 

YES 

□ 

NO 

make. 

and 

!,  how 

wel ' 

Really 

Don't 

Neutral : 

Like 

Really 

Dislike 

Like 

If  I  have  it,  fine 

If  I  don’t  have  it,  fine 

It 

Like  It 

□ 

□ 

□ 

□ 

□ 

4.  Is  the  accuracy  of  the  voice  recognition  system  adequate  enough  to  make  you 
want  to  use  it  in  your  daily  job? 


Very 

Inadequate 

Slightly 

Inadequate 

Neutral 

SI ightly 
Adequate 

Very  Adequate 

□ 

□ 

□ 

in 

□ 

Comparing  voice  input  to  manual  typing 

input,  is  voice 

input: 

Undoubtedly 

Moderately 

The 

Moderately 

Undoubtedly 

Worse 

Worse 

Same 

Better 

Better 

□ 

□ 

a 

□ 

O 

43 


6.  Do  you  consider  yourself  a  slow,  intermediate,  or  fast  typist? 


Slow  Intermediate 

□  n 

How  accurately  do  you  think  you  type? 

Poor  Fair 


□ 


n 


Fast 

[=1 


Quite  Well 
□ 


8.  Did  you  ever  take  a  typing  course? 

Yes  No 

□  a 

9.  What  part  of  the  country,  (USA)  did  you  grow  up  in?  _ 

10.  How  many  years  experience  have  you  had  in  typing  information  into  computers? 
_ years . 

11.  Please  tell  the  experimenter  how  much  and  what  type  of  educational  background 
you  have. 
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H:8 

23:8 

’?:? 

i?:8 

K:g  8:8 

2 

1 

2 

1 

1 

2 

5 

4 

5 

4 

§ 

67.5 

21.0 

2 

2 

2 

2 

2 

1 

1 

3 

73.  0 
19.  0 

17.0 

20.5 

7.5 

24.  0 

17.0 

78.0 

1 

2 

1 

4 

3 

4 

21.0 

1 

2 

2 

4 

77.  5 

17.0 

20.5 

7.5 

24.  0 

17.0 

10.5 

1 

2 

1 

4 

4 

4 

31.5 

2 

2 

2 

1 

32.  0 

13.5 

15.  5 

7.5 

17.0 

17.0 

42.0 

2 

2 

2 

4 

4 

4 

31.5 

2 

2 

1 

3 

32.  0 

13.5 

13.0 

19.0 

7.0 

17.0 

42.0 

2 

2 

2 

4 

3 

4 

21.0 

2 

2 

1 

6 

56. 0 

11.5 

6.5 

19.0 

7.0 

6.5 

48.5 

2 

2 

1 

4 

5 

5 

52.0 

2 

1 

1 

3 

6.  0 

11.5 

6.5 

19.0 

7.0 

6.5 

26.5 

0 

1 

1 

5 

5 

5 

67.5 

2 

3 

1 

1 

32.  0 

5.5 

6.  5 

7.  5 

7.0 

6.5 

10.5 

0 

2 

1 

5 

5 

5 

67.5 

2 

2 

1 

1 

47.  5 

5.5 

6.5 

7.  5 

7.  0 

6.5 

18.5 

2 

2 

1 

5 

5 

5 

67.5 

2 

2 

1 

5 

19.0 

5.5 

6.5 

7.  5 

7.0 

6.5 

77.0 

6 

2 

1 

5 

5 

5 

67.5 

1 

2 

1 

5 

59.  0 

5.5 

6.5 

7.  5 

7.0 

6.5 

63.5 

1 

2 

1 

4 

5 

4 

43.0 

2 

2 

2 

1 

56.  0 

5.5 

6.5 

7.5 

7.0 

6.5 

48.5 

2 

2 

2 

3 

3 

4 

14.5 

1 

1 

1 

1 

41.0 

5.5 

6.5 

7.  5 

7.  0 

6.5 

10.5 

1 

2 

1 

5 

5 

5 

67.5 

2 

2 

1 

3 

32.0 

5.5 

6.5 

7.  5 

7.0 

6.5 

55.0 

1 

2 

1 

4 

4 

5 

43.0 

1 

2 

1 

2 

32.  0 

5.5 

6.5 

7.  5 

7.0 

6.5 

26.5 

1 

1 

1 

3 

3 

3 

10.5 

3 

3 

1 

3 

6~f.  0 

5.5 

6.5 

7.  5 

7.0 

6.5 

1.0 

2 

2 

1 

5 

5 

5 

67.5 

2 

3 

1 

1 

56.  0 

5.5 

6.5 

7.  5 

7.0 

6.5 

55.0 

0 

1 

1 

5 

5 

5 

67.5 

1 

1 

1 

2 

41.0 
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